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Through dynamic testing (the notion that tailored 
testing can be extended to the use o£ & learning oriented approach to 
assessment), analysis vere made of hotr motivational, personality, ,,and 
cognitive style factors interact vith assessment approaches to yield 
performance data* Testing procedures involving simple feedback, 
elaborated feedback, and subject verbalization vere ap|»lied to the 
Havei^ Coloured Progressive ttatrices (cPttt, a test of nonverbal 
intelligence, and selected Piaget measures, and were tested in 
second* and fcurth-grade children from the general population* 
Applicability of the approach vas demonstrated for populations of 
children between 5 and 1p years of age, and of **nornal" intelligence 
(IQy, of learninq handicapped, low 10, and from Anglo, Black, and 
flispanic-'American ethnic groups* Analysis, was made of the interactive 
effects of testing condition with the cognitive style variable, 
impulslvity^reflectivity^ Svideiice vas presented that compensatory 
effects can fee obtained for impulsive jchildr en and jibhildren- of ethnic - — 
minorities utilizing the procedures qf verbalization and elaborated 
feedback* The indication that compensatory effects can be obtained in 
assessing between ethnic group differences mav be useful in reducing 
test bias, but larger samples vith control over factors such as 
soclo-^economic status are necessary before reaching firm conclusions* 
(Authct/HL) 
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Oynamlc Assessment : An Approach Toward fteducing Test Bias 

Abstract 

The bases of the dynamic assessn^nt approach are presented* Tes** 
ting procedures Involving simple feedback^ elaborated feedback^ and 
subject verbalization are applied to the Raven Matric<?s and selected 
Plaget^ measures* Applicability of the approach is demonstrated for 
several different populations of children* These include children of 
••normal" intelligence^ low IQ children, and childrjen from different 
ethnic/racial groups* Analys^is nade of the interactive effects of 
testing condition with the cognitive style variable, imptjlsivity- 
reflectivity* Evidence Is presented for compensatory effects for 
impulsive children and children of ethnic minorities under verbaliza- 
tion and elaborated feedback con^itions*^ The results are discussed in 
; - terms of reducticm of bias in testing. 




Dynamic Assessment: An Approach Toward Reducing Test Bias \ 

A significant concern of our era Is related tfi the con* 
ceptuallzation of what constitutes appropriate assessment of cognitive 
and perceptuaV processes. The debate has focused on key terms such as 
**test' bias" or "test fairness.** Several different approaches^ this 
Issue have been taken. These range from the classical test fairness 
concept of equal prediction elaborated by Cleary (1968),. Cleary* 
Humhreys» Kendrick, and Hesman X^97S), and Eicnorn and Bass (1971 )» to 
criteria involving group menibership and cultural variables ^ 
(Darlington, 1971)» and to the Incorporation of sociological data In- 
arriving at ^erfonnance level (Mercer & Lewis* 1975). 

Recently*' Flaugher^ (1978) has cautioned that test bias can result 
from many factors and Influences and that focus on jast one aspect may 
be ni/oplc and Ignore the breacith of the Issues Involved. He did sug- 
gest» however, that the dfevelopment-of "tailored testing" Would per* 
haps provide a solution to one very Important factor: bias related to 
the atmosphere In which the test Is administered. The notion of 
tailored testing can be extended to the use of a learning oriented ap- 
proach to assessment (Resnick, 1979). Through such an apprjach (re- 
ferred to here as dynamic testing) » analyses can be made of how 
nntlvatlonal » personality* at^d cognitive s1;yle factors Interact with 
assessment approScIies to yield performance^data. This allows for an- 
alysis of bias between and within ethnic and/or cultural groups and 
cl«rtffcatixjn-of-fiKK^dual"and~^roup"44ffeFence&--1n terms- of-psy- — 
chologlcal and Information-professing variables. In addition. 



, dyn^'Ttic assessment provides a basis for modifications in the classroom 
, to incorporate learning characteristics of the child in the strategy 
and/or te/iching methodology used. ^ ■ 

The rationale for the us^ of dynimic testing* and empirical re-^ 
suits of selected studies validating the approach for various groups 
of children will be qutlined. v / 
Bases of Dynamic Testing 

An assumption of ten made in arvsessment of cogrtitive functioning 
is that test scores obtained in the traditional » standardized manner 
serve as adequate estimates of' the cognitive competence of the P^r^on 
-or grjup of persons tested. ^The validity of the assumption is based 
on the notion that performance is a veridical measure of competence. 
This suggests that intra-indivjdual variations in information proces* 
sing» resulting primarily from intellectual and/or personality 
factors* pl^ty^ relatively unimportant roles in tcist outcome. The re^ 
lationship between subject* and task-^independent variables is ignqirtid 
(cf. 4)ettennan» 1979). ^ 

The static test approach and the assumptior^b^hind it has been 
challenged bV several researchers (Brown and Frencft^ 1979; Schmidt* 
197U Guthke» 1977^Haywood» in press). It is argued that testing 
procedures must be developed and applied which ajdequately take intra- 
individual /ariability into account and reduce the potential dis- 
crepancy between measured cognitive functioning and cognitive com- 
petence CBorifner^lindlBi^ 1970). This calU for a d^nimic testing 

approach* where tnodiflcations In testing procedures can be analyzed as 

■ -I 



4\ 



th^ affect perforaance levels and interact with sources of individual 
variation. 

* 

^ In this approach* variations In testing procedure can be built^ 
directly into the test situation itself or implemented through train- 

« * ^ 

ing outside of the actual testing. The latter requires subsequent as^ 
sessri^t of the effects of the training on a transfer test. We favor 
the fom^r: Incorporating the modifications in the procedures Into the 
testing s'^1tuation» for pragmatic and methodological reasons. 
Pragmatically* it Is convenient and directly applii^ble for use by 
psychological practitioners. Methodologically* it avoids problems re- 
lated to the measurement of change (Cronbach and Furby* 1970). ^ 
Testing- Condition's leading to Increased levels of Performance 

. First* several types of commonly used variable^ were isolated 
ft _ ....... 

whIcK can be Incorporated into testing procedures and be Independently 

jnanlpulated (CarUon and Wiedl* 1979). 

^The testing procedures Isolated involve different ,techn1ijues of 
optimizing cognitive functioning. Thus* controlled comfyarison between 
types of approaches often used* ^ut heretofore not systematically an- 
alysed In the dynamic testing approach* was made. The dependent 
measure wAs the Raven Coloured Progressive Matrices (CPM)* a test of 
nonverbal Intelligence (Raven* 1965). The CPM was chosen as it is 
considered to be a relatively ''culture tair^'^test and a good fne<isure 
of general intelligence. The subjects tested were 434 sec(jnd- and 
fourth-grade chidlren from the general population.' The testing con* 
ditlons (C) employed were: 



^ 
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Ci Standard Instructionraccordlng to the procedures outlined 
by Raven. 

Verbalization During and After Solution, which requires the 
child.to describe the main stimulus p^tern prior to searching 
for. the correct distractor and, after a particular distreictor is 
chosen, to explain why he made the choice he did. 

C3 Verbalization After Solution, which Involves the child , 
describing the reasons for his choice after the choice is made^ 

C4 Simple Feedback, where the child Is informed only if he is 
, correct or Incorrect., 

Cg Elaborated Feedback, which Involves, In addition to simple ^ 
feedback, an elaboration ^by the experimenter of theji«asons wtfy 
the chosen distractor was correct or incorrect, the principles 
Involved In the task were pointed out: 

Cg Elaborated' Feedback plus Vertallzatlon During and>'After 

Solution, which is a coinblnatlon of Conditions two and five. 
That Is, it Involves the child's verbalization of the pattern to 
he completed, followed by solution and the child's explanation 
"',___forJ;he reasons for solution, .and elaborated feedback by the ex- 
perimenter informing the child of the correctness of his re- 
sponse and explaining the principles Involved In the task. 

f 

' In all cases^ the iShlld's Initial response* regardless of con- 
d1t1on» was scored. 

For the second-grade children* Cbndltlons 2» 5^ and 6 led tq 
higher levels of performance than were found for Conditions 1» 3« and 
4. The most effective cbgditlons were 5 and 6. Foij the fourth-grade 
children. Conditions Z and 6 led^to Improved performance. A trend in 
.this direction was noted under Condition 5. Thus, three cond'^tlons 
for younger cMldren and two procedures for oldel- children were found 
whiph appeared useful for in^ro7lng"test scores, t.fe., "bringing per-' 
formance closer to actual c<»npetence. 

Beyond ascertaining the main effects of the testing conditions on 
total score CPM performance,' closer analysis of the results was 



'obtained by analyzing the interacti\^e effects of ^ferfomance with dif- 
ferentiations which' th^ CPM allows in administratipn and scoring* The 
test can be administered in either the normal booklet form or in a 
puzzle, broad-form version. In the forrner, c6vert proceses must be 

'used'by the subject as he searches for solution. In the latter, overt 
trial-and-^rror procedures can be used.'. Differentiations 'in the test 
were distinguished through previous factor analysis (Wi&ul and Carl- 
son, 1976). The factors isolet^ were: simple t>at tern completion, 
pattern completion through closure, and reasoning by analogy. (See ^ 
Carroll and Maxwell, 1979 and Carlson and Jensen, 1980.) 
' V_Results of analysis of these differentiations showed that the. ^ 

_ ■ - a, 

puzzle version led to higher levels of performance for the second- 
grade* but not the fourth*grade children. The salierrt Item groups 
were^ simple pattern completion and pattern completion Jhrough closure. 
The. effects of the optimizing testing conditions' could be traced ^ 
mainly to those items requiring abstract thinking. 

• On the basis of these results, two types of studies were con- 
ducted. In one type» the goal was to replicate and extend the^ above 
findings by Investiciating whether or not the' saVlent testing con- 
ditions have the effect of optimizing performance for chlldreiyfrom 
special populations'. .In the other type, in addition to extending 
general applicability, the goal was to assess the compensatpry effect 
which the procedures could'have in reducing differences between par^^ 
"ticular groups of children. 
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In a first study Involving learning handicapped children (Carlson 
^ and W1ed1» 1978)» the same testing psre^dlgm as described for the first 
study was used. .Irf addition* however* the test was readmlnistered 
under to all^^ subjects after a two week .Interval. The^ sample con- 
sisted of 108 special school chldlren, mean age 10.5 y^aiirs^mean 
MISC-IQ 70.8. Kain effect differences were found for-testlng con- ^ 
dition and repeated testing. Conditions/* 5* and 6 led to higher 
performance than found under the other conditions. No testing con- 

* 

dition by repeated testing Interaction was. found. The results aVe 
very similar to those, obtained with. children of average IQ» adding 
further 'evidence for the^-^ffectlveness of the salient conditions. / 

In addition to the main effect found for testing condition,, a 
win effect due to test form (booklel^-puzzle) was detected* with p?r- 
fonnance on the puzzle-form being higher than on the book let -form. 
Further, It was evident that for the puzzle version of the test Con- 

^ ditions 5 and 6» but no Condltljon 2» were effective. 

, Local Ization^of t»^ effects to the subparts oT the CPK revealed 
that testing' coi^ltlons affected performance on the rea'^onlng lUms 
only. Conditions 5 and 6 proved to be the most salient conditions. 

■ The effect of the board-form could be localized to pattern completion 

' and gestalt closure Items* 

- — * ' \ _ < 

The most Important results described In the preceding sections 
are ^summarized In-Table 1. ■ ' 



Insert Table 1 about here 
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"Feedback- ' ^ 
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Xtaii Itequitenientfl 



t 0 r 



T p o r 



t p 0 r 



fte^tilar School 
Child»ti'2'^ 9rM9 

'* Children vith* ti«a«tilng 
OiCficulti«s • 
(at/'g* 4th grads) 



0 
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p « puttezti oompletlon ile«s 
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Perspnitlity a1id Cognitive Style Variables ^ *' 

The above presentation of the dynamic assessment approach has 
■ b?en' concerned primarily- with the validation of the dynamic assessment 
prgcedurfe 'In. l^ems of effects on mean performance scores. Beyond 
this, however, it is Important to Investigate how and to what extent 
learning charatteristic$^(personality and cognitive style) of the 
child-iJEtt.eract-Wlth (lynamic -testing approaches/ ^ 

s Several studVes have stressed the Interaction of learning and 
personality characteristics. Two will be mentioned here that relate 
to one >acf1ciilar ^rsotiallty (style) characteristic:- Itrpulslvlty- ' 
reflectivity. Bush and Dweck (1975) have concludet* that task demands 
InflOende strategics children use who display reflective Information 
processing while Impulsive x:>)11dren seem. to be unaware of the 
Strategies they should match with. the demand of the task. In other 
w0rds» th§y don't adapt strateglfes -to task re()u1reirents. In a Hutfy 
^ relating the dynamic assessi^eht approach tof impulsive children* Wiedl 
(1980)^ hypothesized that the performance of impUlslves could be m^de 
equal to that of reflectives' by modified* more' appropriate testing^ 
" procedures". The.subjects-.were 150 second-grade chilc'en, average age 
/.e yeaVs.. The Matching Fanlliar Figures test (KFF) was administered 



-_to-AU„chli4ceA to_ assess degre e^ Of "IfRpul si vi ty-ref lect i vlty. R 3sed 
on their Wf performance the subjepts were d1vi(led Into two ^/oups» ^ 
Impulslves or reflectlVes. Ihe^classlficatlon was obtained by 
medtifnspllt on the er'rbr'sciire of the test. The CP*' was administered- 

' under -testing-Conditions 1» Z> 5». artd 6. 
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The results stowed that i™pu1s1ves scored lowest o. the CPM u^der . 
standard testl-g condition (Ci). No differences between the 
l^pulslves and reflfctlves we„ detected under Conditions Z and 5. 
That- IS, significant gains in perfonnance were noted for the 
,„puls1ves. but not for" the reflectlves when conditions of verballza- 

' tlon (C;)..nd elaborated feed b-B1^ie5^were-emptoyeWUnde!LCon■ 

d.tlon 6. no c<».pensatory effect was detected for the^^mpulslve 
Children. In'addltlon to the «FF error sco,% classification, children 
were also categorUed as ImpOlslves or reflectlves by a median-spin 
on latency to response. For this classification, no compensatory . 
effects attributable to testing condition were detected In matrices 
perfonnance. 

.„r---".-- with «e,ar^ r.hnic/Racial Ojfferences=_A!alo^ 
.^.Hcan. Black- '-.-i'-'"- Mlsoanlc-Mierlcan Children^ 

The potential c<»,pensatory effects of the dynamic testing ap- 

ppoach in assessing cognitive capabilities of three different ethnic , 
groups. Anglo. Black, and HIspanlcM^rlcan. was studied (Dillon and 
Carlson. 1978). Several in«st1gat1on5 have been carried out com- 
paring- perfomance of these groups on tests like the Raven matrlc^ 
a«l »ar1«,s Plajet tests. Host results <nd1cate that Black children 
. tend to perfom approximately one standard deviation b.low Anglos. 
«1th Hispanic-American perfonnance half-way between. It was hypo- 
thesised that ti.. testing conditions shown to be salient In the pre- 
vious studies would c^pensate for factors which may contribute to the 
relatively low Black and HIspanfc^Amerlcan perfonnance. 
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The tests used were Hatrlces ..d Order of Appe.r.-ce take. fro. . 
.ecertly/developed Pl.set battery (W,.ke,™... 1976). The testl-s 
procedures involved three conditions: Standard (Ci). Verbalization 
0„r1,« and After Solution (C2)..and Verbalization During and After 
solution plus Elaborated Feedback M- ' 

The tasks were administere d to 189 children, representing three 
taerican ethnic groups: Anglo. BUck. and Hispanic-African. The 
subject, ranged fr» five to ten y,ars of age. Randomization and as- 
sig™««t to testing condition was carried out for three age levels: - 
,.•6. 7.8. and 9-10 years. This resulted in a 3.3«3 rand^i zed block 
design. With three levels of test administration, three age levels. 
'.^ three ethnic group classifications. There were seven subjects in 

■ each cell* 

■ . ■ Since results for the Matrices and Order of Appearance tests 

^ tW«-to be^ssenfially-t^e sa.». only those for the Matrices 
«i1T be given. They are su^arized in Table 2. Inspection of the 



Insert Table 2 about here 



5 

li^RJC 



„.an standa,. deviations reveals a ™rked decline in differences i. 
perfor^nce between the groups. The differences ranged fro. ap- 
proximately two-thirds of a standard Aviation between Anglo and Black 
perfor^nce under Condition 1. to essentially zer. un*r Condition 6. 
Although this couVd be attributed to a celling effect for the ol*r 
group (.ear.- of Just over 6 fr™ an 8-ite. test), such an inter- 
proutfon is not plausible fbr th. younger and middle groups (means 
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Table 2 
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Means and Standard Deviations of Matrices for Three Age Groups. 
Btfanlc Groups and Testing Conditions <C). 



•get 5-6 



.95 J.U 2.23 



-^SV-^ 71.$--^2, 43 
.98 1.70 2.23 



age ; 7-8 



aee;- 9^10 



3. 43<— > 2 . 43«— > 4 . 29 
2.37 1.51 .95 



2.144->2.73« — >4.00 
.90 1.50 l.Al 



4.14< — >6.00<^ ->6.29 



2.|0 



1.73 



1.60 



i.29tf — 55;7if- ->6.14- 
2.29 1.38 1.07 



^ Hack 



.57<->.86'-<— >2.00 
.79 .85 1.83 



1.43^->2.71«^4.14 
1.40 1.60_ . .90 



2.71«— >5.72<->6.00 
1.38 . 1.25 ^ 1.00 




— ■ dlffeitenca not significant 
,^ ^ dlf fetience significant 
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around 2 and 4 respectively). It is interesting that for all age 
groupings, regardless of ethnk classification* significant increases 
In performance were found under Condition 6 as contrasted with Con- 
dition 1. In addition, the verbalization condition tC2) was ef- 

•r 

fectlve only for the older age groups* again, regardless of ethnic 
classification* Apparently* for these groups the appropriate verbal 
mediation strategies are available. 
Summary 

The approach of applying optimizing testing procedures to the 
nomal population as well as to special groups of children was de-^ 
monstrated. Restrictions and differentiations In the effects of test- 
ing procedures* test fomat* and task demands were . -. The ap- 
pllcablllty of certain testing procedures for purposes of assessing 
Intellectual performance of nomal and special school (learning handi- 
capped* low IQ) children was shown* This Indicates that specific 
foms of performance optimization can lead to more accurate* thus 
fairer* assessment of Intellectual functioning* It was demonstrated 
that specific dysfunctional individual characteristics such as 
Impu^slvity can be compensated for under certain testing conditions* 
This underscores the necessity of critically analyzing how varlatlpns 
In testing procedure cdn Interact with certain Indivlxlual differences. 
The Implication here Is that the practitioner must be aware of these 
Interactions and under which circunstances performance m^y be 
Increased for sofne children^ while decreased for others* 

The Indication that compensatory effects can be obtained in as- 
ses.sing between ethnic group differences Is of practical and' 

.16 



theoretical significance. Extension of the approach by employing 
other ^asures of Level 11 intelligence (Jensen. 1973) to larger 
samples with control over factors such as soclo-'economic status ,s 
necessary before firm conclusions can be reached, however. 
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